Strengths, weaknesses, opportunities and threats of using AI-enabled technology in sleep medicine: a commentary.

Over the past few years, artificial intelligence (AI) has emerged as a powerful tool used to efficiently automate several tasks across multiple domains. Sleep medicine is perfectly positioned to leverage this tool due to the wealth of physiological signals obtained through sleep studies or sleep tracking devices and abundance of accessible clinical data through electronic medical records. However, caution must be applied when utilizing AI, due to intrinsic challenges associated with novel technology. The Artificial Intelligence in Sleep Medicine committee of the American Academy of Sleep Medicine (AASM) reviews advancements in AI within the sleep medicine field. In this article, the Artificial Intelligence in Sleep Medicine committee members provide a commentary on the scope of AI technology in sleep medicine. The commentary identifies three pivotal areas in sleep medicine which can benefit from AI technologies: clinical care, lifestyle management and population health management. This article provides a detailed analysis of the strengths, weaknesses, opportunities, and threats associated with using AI enabled technologies in each pivotal area. Finally, the article broadly reviews barriers and challenges associated with using AI enabled technologies and offers possible solutions.

Prediction of Effectiveness and Toxicities of Immune Checkpoint Inhibitors Using Real-World Patient Data.

PURPOSE: Although immune checkpoint inhibitors (ICIs) have improved outcomes in certain patients with cancer, they can also cause life-threatening immunotoxicities. Predicting immunotoxicity risks alongside response could provide a personalized risk-benefit profile, inform therapeutic decision making, and improve clinical trial cohort selection. We aimed to build a machine learning (ML) framework using routine electronic health record (EHR) data to predict hepatitis, colitis, pneumonitis, and 1-year overall survival. METHODS: Real-world EHR data of more than 2,200 patients treated with ICI through December 31, 2018, were used to develop predictive models. Using a prediction time point of ICI initiation, a 1-year prediction time window was applied to create binary labels for the four outcomes for each patient. Feature engineering involved aggregating laboratory measurements over appropriate time windows (60-365 days). Patients were randomly partitioned into training (80%) and test (20%) sets. Random forest classifiers were developed using a rigorous model development framework. RESULTS: The patient cohort had a median age of 63 years and was 61.8% male. Patients predominantly had melanoma (37.8%), lung cancer (27.3%), or genitourinary cancer (16.4%). They were treated with PD-1 (60.4%), PD-L1 (9.0%), and CTLA-4 (19.7%) ICIs. Our models demonstrate reasonably strong performance, with AUCs of 0.739, 0.729, 0.755, and 0.752 for the pneumonitis, hepatitis, colitis, and 1-year overall survival models, respectively. Each model relies on an outcome-specific feature set, though some features are shared among models. CONCLUSION: To our knowledge, this is the first ML solution that assesses individual ICI risk-benefit profiles based predominantly on routine structured EHR data. As such, use of our ML solution will not require additional data collection or documentation in the clinic.

Use of noisy labels as weak learners to identify incompletely ascertainable outcomes: A Feasibility study with opioid-induced respiratory depression.

Objective: Assigning outcome labels to large observational data sets in a timely and accurate manner, particularly when outcomes are rare or not directly ascertainable, remains a significant challenge within biomedical informatics. We examined whether noisy labels generated from subject matter experts' heuristics using heterogenous data types within a data programming paradigm could provide outcomes labels to a large, observational data set. We chose the clinical condition of opioid-induced respiratory depression for our use case because it is rare, has no administrative codes to easily identify the condition, and typically requires at least some unstructured text to ascertain its presence. Materials and methods: Using de-identified electronic health records of 52,861 post-operative encounters, we applied a data programming paradigm (implemented in the Snorkel software) for the development of a machine learning classifier for opioid-induced respiratory depression. Our approach included subject matter experts creating 14 labeling functions that served as noisy labels for developing a probabilistic Generative model. We used probabilistic labels from the Generative model as outcome labels for training a Discriminative model on the source data. We evaluated performance of the Discriminative model with a hold-out test set of 599 independently-reviewed patient records. Results: The final Discriminative classification model achieved an accuracy of 0.977, an F1 score of 0.417, a sensitivity of 1.0, and an AUC of 0.988 in the hold-out test set with a prevalence of 0.83% (5/599). Discussion: All of the confirmed Cases were identified by the classifier. For rare outcomes, this finding is encouraging because it reduces the number of manual reviews needed by excluding visits/patients with low probabilities. Conclusion: Application of a data programming paradigm with expert-informed labeling functions might have utility for phenotyping clinical phenomena that are not easily ascertainable from highly-structured data.

Use of Noisy Labels as Weak Learners to Identify Incompletely Ascertainable Outcomes: A Feasibility Study with Opioid-Induced Respiratory Depression.

Objective: Assigning outcome labels to large observational data sets in a timely and accurate manner, particularly when outcomes are rare or not directly ascertainable, remains a significant challenge within biomedical informatics. We examined whether noisy labels generated from subject matter experts' heuristics using heterogenous data types within a data programming paradigm could provide outcomes labels to a large, observational data set. We chose the clinical condition of opioid-induced respiratory depression for our use case because it is rare, has no administrative codes to easily identify the condition, and typically requires at least some unstructured text to ascertain its presence. Materials and Methods: Using de-identified electronic health records of 52,861 post-operative encounters, we applied a data programming paradigm (implemented in the Snorkel software) for the development of a machine learning classifier for opioid-induced respiratory depression. Our approach included subject matter experts creating 14 labeling functions that served as noisy labels for developing a probabilistic Generative model. We used probabilistic labels from the Generative model as outcome labels for training a Discriminative model on the source data. We evaluated performance of the Discriminative model with a hold-out test set of 599 independently-reviewed patient records. Results: The final Discriminative classification model achieved an accuracy of 0.977, an F1 score of 0.417, a sensitivity of 1.0, and an AUC of 0.988 in the hold-out test set with a prevalence of 0.83% (5/599). Discussion: All of the confirmed Cases were identified by the classifier. For rare outcomes, this finding is encouraging because it reduces the number of manual reviews needed by excluding visits/patients with low probabilities. Conclusion: Application of a data programming paradigm with expert-informed labeling functions might have utility for phenotyping clinical phenomena that are not easily ascertainable from highly-structured data.

Scalable Incident Detection via Natural Language Processing and Probabilistic Language Models.

Post marketing safety surveillance depends in part on the ability to detect concerning clinical events at scale. Spontaneous reporting might be an effective component of safety surveillance, but it requires awareness and understanding among healthcare professionals to achieve its potential. Reliance on readily available structured data such as diagnostic codes risk under-coding and imprecision. Clinical textual data might bridge these gaps, and natural language processing (NLP) has been shown to aid in scalable phenotyping across healthcare records in multiple clinical domains. In this study, we developed and validated a novel incident phenotyping approach using unstructured clinical textual data agnostic to Electronic Health Record (EHR) and note type. It's based on a published, validated approach (PheRe) used to ascertain social determinants of health and suicidality across entire healthcare records. To demonstrate generalizability, we validated this approach on two separate phenotypes that share common challenges with respect to accurate ascertainment: 1) suicide attempt; 2) sleep-related behaviors. With samples of 89,428 records and 35,863 records for suicide attempt and sleep-related behaviors, respectively, we conducted silver standard (diagnostic coding) and gold standard (manual chart review) validation. We showed Area Under the Precision-Recall Curve of ∼ 0.77 (95% CI 0.75-0.78) for suicide attempt and AUPR ∼ 0.31 (95% CI 0.28-0.34) for sleep-related behaviors. We also evaluated performance by coded race and demonstrated differences in performance by race were dissimilar across phenotypes and require algorithmovigilance and debiasing prior to implementation.

Crowdsourcing Skin Demarcations of Chronic Graft-Versus-Host Disease in Patient Photographs: Training Versus Performance Study.

BACKGROUND: Chronic graft-versus-host disease (cGVHD) is a significant cause of long-term morbidity and mortality in patients after allogeneic hematopoietic cell transplantation. Skin is the most commonly affected organ, and visual assessment of cGVHD can have low reliability. Crowdsourcing data from nonexpert participants has been used for numerous medical applications, including image labeling and segmentation tasks. OBJECTIVE: This study aimed to assess the ability of crowds of nonexpert raters-individuals without any prior training for identifying or marking cGHVD-to demarcate photos of cGVHD-affected skin. We also studied the effect of training and feedback on crowd performance. METHODS: Using a Canfield Vectra H1 3D camera, 360 photographs of the skin of 36 patients with cGVHD were taken. Ground truth demarcations were provided in 3D by a trained expert and reviewed by a board-certified dermatologist. In total, 3000 2D images (projections from various angles) were created for crowd demarcation through the DiagnosUs mobile app. Raters were split into high and low feedback groups. The performances of 4 different crowds of nonexperts were analyzed, including 17 raters per image for the low and high feedback groups, 32-35 raters per image for the low feedback group, and the top 5 performers for each image from the low feedback group. RESULTS: Across 8 demarcation competitions, 130 raters were recruited to the high feedback group and 161 to the low feedback group. This resulted in a total of 54,887 individual demarcations from the high feedback group and 78,967 from the low feedback group. The nonexpert crowds achieved good overall performance for segmenting cGVHD-affected skin with minimal training, achieving a median surface area error of less than 12% of skin pixels for all crowds in both the high and low feedback groups. The low feedback crowds performed slightly poorer than the high feedback crowd, even when a larger crowd was used. Tracking the 5 most reliable raters from the low feedback group for each image recovered a performance similar to that of the high feedback crowd. Higher variability between raters for a given image was not found to correlate with lower performance of the crowd consensus demarcation and cannot therefore be used as a measure of reliability. No significant learning was observed during the task as more photos and feedback were seen. CONCLUSIONS: Crowds of nonexpert raters can demarcate cGVHD images with good overall performance. Tracking the top 5 most reliable raters provided optimal results, obtaining the best performance with the lowest number of expert demarcations required for adequate training. However, the agreement amongst individual nonexperts does not help predict whether the crowd has provided an accurate result. Future work should explore the performance of crowdsourcing in standard clinical photos and further methods to estimate the reliability of consensus demarcations.

Development of a machine learning model to predict intraoperative transfusion and guide type and screen ordering.

STUDY OBJECTIVE: To develop an algorithm to predict intraoperative Red Blood Cell (RBC) transfusion from preoperative variables contained in the electronic medical record of our institution, with the goal of guiding type and screen ordering. DESIGN: Machine Learning model development on retrospective single-center hospital data. SETTING: Preoperative period and operating room. PATIENTS: The study included patients ≥18 years old who underwent surgery during 2019-2022 and excluded those who refused transfusion, underwent emergency surgery, or surgery for organ donation after cardiac or brain death. INTERVENTION: Prediction of intraoperative transfusion vs. no intraoperative transfusion. MEASUREMENTS: The outcome variable was intraoperative transfusion of RBCs. Predictive variables were surgery, surgeon, anesthesiologist, age, sex, body mass index, race or ethnicity, preoperative hemoglobin (g/dL), partial thromboplastin time (s), platelet count x 109 per liter, and prothrombin time. We compared the performances of seven machine learning algorithms. After training and optimization on the 2019-2021 dataset, model thresholds were set to the current institutional performance level of sensitivity (93%). To qualify for comparison, models had to maintain clinically relevant sensitivity (>90%) when predicting on 2022 data; overall accuracy was the comparative metric. MAIN RESULTS: Out of 100,813 cases that met study criteria from 2019 to 2021, intraoperative transfusion occurred in 5488 (5.4%) of cases. The LightGBM model was the highest performing algorithm in external temporal validity experiments, with overall accuracy of (76.1%) [95% confidence interval (CI), 75.6-76.5], while maintaining clinically relevant sensitivity of (91.2%) [95% CI, 89.8-92.5]. If type and screens were ordered based upon the LightGBM model, the predicted type and screen to transfusion ratio would improve from 8.4 to 5.1. CONCLUSIONS: Machine learning approaches are feasible in predicting intraoperative transfusion from preoperative variables and may improve preoperative type and screen ordering practices when incorporated into the electronic health record.

Accelerated curation of checkpoint inhibitor-induced colitis cases from electronic health records.

Objective: Automatically identifying patients at risk of immune checkpoint inhibitor (ICI)-induced colitis allows physicians to improve patientcare. However, predictive models require training data curated from electronic health records (EHR). Our objective is to automatically identify notes documenting ICI-colitis cases to accelerate data curation. Materials and Methods: We present a data pipeline to automatically identify ICI-colitis from EHR notes, accelerating chart review. The pipeline relies on BERT, a state-of-the-art natural language processing (NLP) model. The first stage of the pipeline segments long notes using keywords identified through a logistic classifier and applies BERT to identify ICI-colitis notes. The next stage uses a second BERT model tuned to identify false positive notes and remove notes that were likely positive for mentioning colitis as a side-effect. The final stage further accelerates curation by highlighting the colitis-relevant portions of notes. Specifically, we use BERT's attention scores to find high-density regions describing colitis. Results: The overall pipeline identified colitis notes with 84% precision and reduced the curator note review load by 75%. The segment BERT classifier had a high recall of 0.98, which is crucial to identify the low incidence (<10%) of colitis. Discussion: Curation from EHR notes is a burdensome task, especially when the curation topic is complicated. Methods described in this work are not only useful for ICI colitis but can also be adapted for other domains. Conclusion: Our extraction pipeline reduces manual note review load and makes EHR data more accessible for research.

Objective Pharyngeal Phenotyping in Obstructive Sleep Apnea With High-Resolution Manometry.

OBJECTIVE: Drug-induced sleep endoscopy (DISE) is a commonly used diagnostic tool for surgical procedural selection in obstructive sleep apnea (OSA), but it is expensive, subjective, and requires sedation. Here we present an initial investigation of high-resolution pharyngeal manometry (HRM) for upper airway phenotyping in OSA, developing a software system that reliably predicts pharyngeal sites of collapse based solely on manometric recordings. STUDY DESIGN: Prospective cross-sectional study. SETTING: An academic sleep medicine and surgery practice. METHODS: Forty participants underwent simultaneous HRM and DISE. A machine learning algorithm was constructed to estimate pharyngeal level-specific severity of collapse, as determined by an expert DISE reviewer. The primary outcome metrics for each level were model accuracy and F1-score, which balances model precision against recall. RESULTS: During model training, the average F1-score across all categories was 0.86, with an average weighted accuracy of 0.91. Using a holdout test set of 9 participants, a K-nearest neighbor model trained on 31 participants attained an average F1-score of 0.96 and an average accuracy of 0.97. The F1-score for prediction of complete concentric palatal collapse was 0.86. CONCLUSION: Our findings suggest that HRM may enable objective and dynamic mapping of the pharynx, opening new pathways toward reliable and reproducible assessment of this complex anatomy in sleep.

Feasibility and Short-Term Effects of a Multi-Component Emergency Department Blood Pressure Intervention: A Pilot Randomized Trial.

Background Emergency department (ED) visits can be opportunities to address uncontrolled hypertension. We sought to compare short-term blood pressure measures between the Vanderbilt Emergency Room Bundle (VERB) intervention and usual care plus education. Methods and Results We conducted a randomized trial of 206 adult patients with hypertension and elevated systolic blood pressure (SBP) presenting to 2 urban emergency departments in Tennessee, USA. The VERB intervention included educational materials, a brief motivational interview, pillbox, primary care engagement letter, pharmacy resources, and 45 days of informational and reminder text messages. The education arm received a hypertension pamphlet. After 78 participants were enrolled, text messages requested confirmation of receipt. The primary clinical outcome was 30-day SBP. The median 30-day SBP was 122 and 126 mm Hg in the VERB and education arms, respectively. We estimated the mean 30-day SBP to be 3.98 mm Hg lower in the VERB arm (95% CI, -2.44 to 10.4; P=0.22). Among participants enrolled after text messages were adapted, the respective median SBPs were 121 and 130 mm Hg, and we estimated the mean 30-day SBP to be 8.57 mm Hg lower in the VERB arm (95% CI, 0.98‒16.2; P=0.027). In this subgroup, the median response rate to VERB text messages was 56% (interquartile range, [26%‒80%]). Conclusions This pilot study demonstrated feasibility and found an improvement in SBP for the subgroup for whom interactive messages were featured. Future studies should evaluate the role of interactive text messaging as part of a comprehensive emergency department intervention to improve blood pressure control. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT02672787.

Classifying Infection Risk Following Pediatric Cardiac Surgery.

Postoperative infections frequently complicate pediatric cardiac surgery, increasing morbidity and cost. If high risk patients could be identified early, preventive measures could mitigate infection risk. In this study, we used structured health data to generate a cohort of pediatric cardiac surgery cases from a single center and used billing codes to assign outcomes for postoperative sepsis, bacteremia, necrotizing enterocolitis, and a composite outcome. We subsequently validated these outcomes manually using clinical notes and culture data. Using this cohort of 2080 surgeries, we trained models to classify the risk of postoperative infections using logistic regression and several machine learning methods. We compared the performance of the models trained on the validated outcomes to those trained on unvalidated outcomes. Manual validation revealed low accuracy of diagnosis codes as classifiers of postoperative infections. Despite significant differences in outcome assignments, similar model performance was achieved using unvalidated and validated outcomes.

Semi-Automated Data Curation from Biomedical Literature.

Data curation is a bottleneck for many informatics pipelines. A specific example of this is aggregating data from preclinical studies to identify novel genetic pathways for atherosclerosis in humans. This requires extracting data from published mouse studies such as the perturbed gene and its impact on lesion sizes and plaque inflammation, which is non-trivial. Curation efforts are resource-heavy, with curators manually extracting data from hundreds of publications. In this work, we describe the development of a semi-automated curation tool to accelerate data extraction. We use natural language processing (NLP) methods to auto-populate a web-based form which is then reviewed by a curator. We conducted a controlled user study to evaluate the curation tool. Our NLP model has a 70% accuracy on categorical fields and our curation tool accelerates task completion time by 49% compared to manual curation.

Systems Approach to Integrating Preclinical Apolipoprotein E-Knockout Investigations Reveals Novel Etiologic Pathways and Master Atherosclerosis Network in Humans.

OBJECTIVE: Animal models of atherosclerosis are used extensively to interrogate molecular mechanisms in serial fashion. We tested whether a novel systems biology approach to integration of preclinical data identifies novel pathways and regulators in human disease. Approach and Results: Of 716 articles published in ATVB from 1995 to 2019 using the apolipoprotein E knockout mouse to study atherosclerosis, data were extracted from 360 unique studies in which a gene was experimentally perturbed to impact plaque size or composition and analyzed using Ingenuity Pathway Analysis software. TREM1 (triggering receptor expressed on myeloid cells) signaling and LXR/RXR (liver X receptor/retinoid X receptor) activation were identified as the top atherosclerosis-associated pathways in mice (both P<1.93×10-4, TREM1 implicated early and LXR/RXR in late atherogenesis). The top upstream regulatory network in mice (sc-58125, a COX2 inhibitor) linked 64.0% of the genes into a single network. The pathways and networks identified in mice were interrogated by testing for associations between the genetically predicted gene expression of each mouse pathway-identified human homolog with clinical atherosclerosis in a cohort of 88 660 human subjects. Homologous human pathways and networks were significantly enriched for gene-atherosclerosis associations (empirical P<0.01 for TREM1 and LXR/RXR pathways and COX2 network). This included 12(60.0%) TREM1 pathway genes, 15(53.6%) LXR/RXR pathway genes, and 67(49.3%) COX2 network genes. Mouse analyses predicted, and human study validated, the strong association of COX2 expression (PTGS2) with increased likelihood of atherosclerosis (odds ratio, 1.68 per SD of genetically predicted gene expression; P=1.07×10-6). CONCLUSIONS: PRESCIANT (Preclinical Science Integration and Translation) leverages published preclinical investigations to identify high-confidence pathways, networks, and regulators of human disease.

User preferences for and engagement with text messages to support antihypertensive medication adherence: Findings from a pilot study evaluating an emergency department-based behavioral intervention.

OBJECTIVE: We examined users' preferences for and engagement with text messages delivered as part of an emergency department (ED)-based intervention to improve antihypertensive medication adherence. METHODS: We recruited ED patients with elevated blood pressure for a pilot randomized trial evaluating a medication adherence intervention with text messages. Intervention participants chose text content and frequency, received texts for 45 days, and completed a feedback survey. We defined engagement via responses to texts. We examined participant characteristics associated with text preferences, engagement, and feedback. RESULTS: Participants (N = 101) were 57% female and 46% non-White. Most participants (71%) chose to receive both reminder and informational texts; 94% chose reminder texts once per day and 97% chose informational texts three times per week. Median text message response rate was 56% (IQR 26-80%). Participants who were Black (p < 0.01), had lower income (p = 0.03), or had lower medication adherence (p < 0.01) rated the program as more helpful and wanted additional functionalities for adherence support. CONCLUSIONS AND PRACTICE IMPLICATIONS: While overall engagement was modest, participants at risk of worse health outcomes expressed more value and interest in the program. Findings inform the design of text messaging interventions for antihypertensive medication adherence and support targeting vulnerable patients to reduce health disparities. CLINICAL TRIALS REGISTRATION: NCT02672787.

Leveraging medical context to recommend semantically similar terms for chart reviews.

BACKGROUND: Information retrieval (IR) help clinicians answer questions posed to large collections of electronic medical records (EMRs), such as how best to identify a patient's cancer stage. One of the more promising approaches to IR for EMRs is to expand a keyword query with similar terms (e.g., augmenting cancer with mets). However, there is a large range of clinical chart review tasks, such that fixed sets of similar terms is insufficient. Current language models, such as Bidirectional Encoder Representations from Transformers (BERT) embeddings, do not capture the full non-textual context of a task. In this study, we present new methods that provide similar terms dynamically by adjusting with the context of the chart review task. METHODS: We introduce a vector space for medical-context in which each word is represented by a vector that captures the word's usage in different medical contexts (e.g., how frequently cancer is used when ordering a prescription versus describing family history) beyond the context learned from the surrounding text. These vectors are transformed into a vector space for customizing the set of similar terms selected for different chart review tasks. We evaluate the vector space model with multiple chart review tasks, in which supervised machine learning models learn to predict the preferred terms of clinically knowledgeable reviewers. To quantify the usefulness of the predicted similar terms to a baseline of standard word2vec embeddings, we measure (1) the prediction performance of the medical-context vector space model using the area under the receiver operating characteristic curve (AUROC) and (2) the labeling effort required to train the models. RESULTS: The vector space outperformed the baseline word2vec embeddings in all three chart review tasks with an average AUROC of 0.80 versus 0.66, respectively. Additionally, the medical-context vector space significantly reduced the number of labels required to learn and predict the preferred similar terms of reviewers. Specifically, the labeling effort was reduced to 10% of the entire dataset in all three tasks. CONCLUSIONS: The set of preferred similar terms that are relevant to a chart review task can be learned by leveraging the medical context of the task.

Classification and analysis of asynchronous communication content between care team members involved in breast cancer treatment.

OBJECTIVE: A growing research literature has highlighted the work of managing and triaging clinical messages as a major contributor to professional exhaustion and burnout. The goal of this study was to discover and quantify the distribution of message content sent among care team members treating patients with breast cancer. MATERIALS AND METHODS: We analyzed nearly two years of communication data from the electronic health record (EHR) between care team members at Vanderbilt University Medical Center. We applied natural language processing to perform sentence-level annotation into one of five information types: clinical, medical logistics, nonmedical logistics, social, and other. We combined sentence-level annotations for each respective message. We evaluated message content by team member role and clinic activity. RESULTS: Our dataset included 81 857 messages containing 613 877 sentences. Across all roles, 63.4% and 21.8% of messages contained logistical information and clinical information, respectively. Individuals in administrative or clinical staff roles sent 81% of all messages containing logistical information. There were 33.2% of messages sent by physicians containing clinical information-the most of any role. DISCUSSION AND CONCLUSION: Our results demonstrate that EHR-based asynchronous communication is integral to coordinate care for patients with breast cancer. By understanding the content of messages sent by care team members, we can devise informatics initiatives to improve physicians' clerical burden and reduce unnecessary interruptions.

Delirium Variability is Influenced by the Sound Environment (DEVISE Study): How Changes in the Intensive Care Unit soundscape affect delirium incidence.

Quantitative data on the sensory environment of intensive care unit (ICU) patients and its potential link to increased risk of delirium is limited. We examined whether higher average sound and light levels in ICU environments are associated with delirium incidence. Over 111 million sound and light measurements from 143 patient stays in the surgical and trauma ICUs were collected using Quietyme® (Neshkoro, Wisconsin) sensors from May to July 2018 and analyzed. Sensory data were grouped into time of day, then normalized against their ICU environments, with Confusion Assessment Method (CAM-ICU) scores measured each shift. We then performed logistic regression analysis, adjusting for possible confounding variables. Lower morning sound averages (8 am-12 pm) (OR = 0.835, 95% OR CI = [0.746, 0.934], p = 0.002) and higher daytime sound averages (12 pm-6 pm) (OR = 1.157, 95% OR CI = [1.036, 1.292], p = 0.011) were associated with an increased odds of delirium incidence, while nighttime sound averages (10 pm-8 am) (OR = 0.990, 95% OR CI = [0.804, 1.221], p = 0.928) and the ICU light environment did not show statistical significance. Our results suggest an association between the ICU soundscape and the odds of developing delirium. This creates a future paradigm for studies of the ICU soundscape and lightscape.

Phenotyping coronavirus disease 2019 during a global health pandemic: Lessons learned from the characterization of an early cohort.

From the start of the coronavirus disease 2019 (COVID-19) pandemic, researchers have looked to electronic health record (EHR) data as a way to study possible risk factors and outcomes. To ensure the validity and accuracy of research using these data, investigators need to be confident that the phenotypes they construct are reliable and accurate, reflecting the healthcare settings from which they are ascertained. We developed a COVID-19 registry at a single academic medical center and used data from March 1 to June 5, 2020 to assess differences in population-level characteristics in pandemic and non-pandemic years respectively. Median EHR length, previously shown to impact phenotype performance in type 2 diabetes, was significantly shorter in the SARS-CoV-2 positive group relative to a 2019 influenza tested group (median 3.1 years vs 8.7; Wilcoxon rank sum P = 1.3e-52). Using three phenotyping methods of increasing complexity (billing codes alone and domain-specific algorithms provided by an EHR vendor and clinical experts), common medical comorbidities were abstracted from COVID-19 EHRs, defined by the presence of a positive laboratory test (positive predictive value 100%, recall 93%). After combining performance data across phenotyping methods, we observed significantly lower false negative rates for those records billed for a comprehensive care visit (p = 4e-11) and those with complete demographics data recorded (p = 7e-5). In an early COVID-19 cohort, we found that phenotyping performance of nine common comorbidities was influenced by median EHR length, consistent with previous studies, as well as by data density, which can be measured using portable metrics including CPT codes. Here we present those challenges and potential solutions to creating deeply phenotyped, acute COVID-19 cohorts.

Extracting Medical Information from Paper COVID-19 Assessment Forms.

OBJECTIVE: This study examines the validity of optical mark recognition, a novel user interface, and crowdsourced data validation to rapidly digitize and extract data from paper COVID-19 assessment forms at a large medical center. METHODS: An optical mark recognition/optical character recognition (OMR/OCR) system was developed to identify fields that were selected on 2,814 paper assessment forms, each with 141 fields which were used to assess potential COVID-19 infections. A novel user interface (UI) displayed mirrored forms showing the scanned assessment forms with OMR results superimposed on the left and an editable web form on the right to improve ease of data validation. Crowdsourced participants validated the results of the OMR system. Overall error rate and time taken to validate were calculated. A subset of forms was validated by multiple participants to calculate agreement between participants. RESULTS: The OMR/OCR tools correctly extracted data from scanned forms fields with an average accuracy of 70% and median accuracy of 78% when the OMR/OCR results were compared with the results from crowd validation. Scanned forms were crowd-validated at a mean rate of 157 seconds per document and a volume of approximately 108 documents per day. A randomly selected subset of documents was reviewed by multiple participants, producing an interobserver agreement of 97% for documents when narrative-text fields were included and 98% when only Boolean and multiple-choice fields were considered. CONCLUSION: Due to the COVID-19 pandemic, it may be challenging for health care workers wearing personal protective equipment to interact with electronic health records. The combination of OMR/OCR technology, a novel UI, and crowdsourcing data-validation processes allowed for the efficient extraction of a large volume of paper medical documents produced during the COVID-19 pandemic.